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1  Summary 


The  aim  of  this  research  has  been  to  develop  quantitative  tools  for  high  resolution  signal  analysis 
and  system  modeling.  These  consist  of  theory  and  software.  An  important  contribution  has  been 
to  introduce  a  family  of  distance  metrics  to  be  used  for  comparing  power  spectral  distributions 
in  multivariable  time-series.  These  metrics  allow  quantitative  assessment  of  uncertainty  and  of 
resolution  in  spectral  analysis.  They  also  provide  a  geometric  substrate  for  new  techniques  in 
filtering  and  estimation.  More  specifically,  the  concepts  of  geodesics  and  that  of  geometric  means, 
have  provided  models  for  time- varying  spectra  and  for  ways  to  aggregate  information  from  disparate 
sources,  respectively.  A  description  of  concepts  and  of  the  software  that  emerged  is  included  and 
maintained  at  the  Pi’s  website  (http://www.ece.umn.edu/~georgiou/files/software.html). 


2  Research  Publications  (2009-2011) 

2.1  Journal  publications 

1.  T.T.Georgiou,  J.  Karlsson,  and  S.  Takyar,  “Metrics  for  power  spectra:  an  axiomatic  approach,” 
IEEE  Trans,  on  Signal  Processing,  57(3):  859  -  867,  2009. 

2.  V.L.  Bageshwar,  D.  Gebre-Egziabher,  W.L.  Garrard,  and  T.T.  Georgiou,  Stochastic  Observ¬ 
ability  Test  for  Discrete-Time  Kalman  Filters,  AIAA  Journal  of  Guidance,  Control,  and  Dy¬ 
namics,  32(4):  1356-1370,  2009. 

3.  T.T.  Georgiou  and  M.C.  Smith,  “Feedback  Control  and  the  Arrow  of  Time,”  International 
Journal  of  Control,  83(7):  1325  1338,  2010. 

4.  J. Karlsson,  T.T.  Georgiou,  and  A.  Lindquist,  “The  Inverse  Problem  of  Analytic  Interpola¬ 
tion  with  Degree  Constraint  and  Weight  Selection  for  Control  Synthesis,”  IEEE  Trans,  on 
Automatic  Control,  55(2):  405-418,  2010. 

5.  Mir  Shahrouz  Takyar,  and  T.T.  Georgiou,  “Multivariable  analytic  interpolation  with  a  degree 
constraint,”  IEEE  Trans,  on  Automatic  Control,  55(5):  1057-1088,  2010. 

6.  F.  Carli  and  T.T.  Georgiou  “On  the  Covariance  Completion  Problem  under  a  Circulant  Struc¬ 
ture,”  IEEE  Trans,  on  Automatic  Control,  56(4):  918  -  922,  2011. 

7.  Xianhua  Jiang,  Zhi-Quan  (Tom)  Luo  and  Tryphon  T.  Georgiou,  “Geometric  Methods  for 
Spectral  Analysis,”  IEEE  Trans,  on  Signal  Processing,  60(3):  1064-1074,  2012. 

8.  Xianhua  Jiang,  Lipeng  Ning,  and  Tryphon  T.  Georgiou,  “Distances  and  Riemannian  metrics 
for  multivariate  spectral  densities,”  IEEE  Trans,  on  Automatic  Control ,  to  appear  in  2012, 
(http: //http:/ /arxiv.org/abs/1107.1345vl) 

9.  T.T.  Georgiou  and  A.  Lindquist,  “The  Separation  Principle  in  Stochastic  Control,  Redux,” 
under  review,  (http://arxiv.org/abs/1103.3005) 

10.  Lipeng  Ning,  Xianhua  Jiang  and  Tryphon  Georgiou,  “Geometric  methods  for  estimation  of 
structured  covariances,”  under  review  (http://arxiv.org/abs/1110.3695) 
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11.  J.Karlsson  and  T.T.  Georgiou,  “Uncertainty  bounds  for  spectral  estimation,”  under  review 
(http://arxiv.org/401938). 

2.2  Book  chapters 

12.  T.T.  Georgiou  and  A.  Tannenbaum,  “Sparse  Blind  Source  Deconvolution  with  Application  to 
High  Resolution  Frequency  Analysis” ,  in  Three  Decades  of  Progress  in  Systems  and  Control , 
Hu,  X.;  Jonsson,  U.;  Wahlberg,  B.;  Ghosh,  B.  (Eds.),  Springer-Verlag,  2009. 

13.  T.T.  Georgiou  and  A.  Tannenbaum,  “Sparse  Blind  Source  Deconvolution  via  ^i-norm  Optimiza¬ 
tion"  ,  in  Perspectives  in  Mathematical  System  Theory ,  Control,  and  Signal  Processing ,  J.C. 
Willems,  S.  Hara,  Y.  Ohta,  and  H.  Fujioka  (Eds.),  pages  321-330,  Springer-Verlag,  2010. 

14.  E.  Tannenbaum,  T.T.  Georgiou,  and  A.  Tannenbaum,  “Optimal  Mass  Transport  for  Problems 
in  Control,  Statistical  Estimation,  and  Image  Analysis,”  Srpinger-Verlag,  to  appear,  2012. 

15.  M.S.  Takyar  and  T.T.  Georgiou,  “Fractional-order  systems  and  the  internal  model  principle,” 
Springer-Verlag,  to  appear,  2012. 

2.3  Conference  proceedings 

16.  K.  Georgiou  and  T.T.  Georgiou,  “Graceful  switching  in  hybrid  models,”  48th  IEEE  Conference 
on  Decision  and  Control ,  Shanghai,  China,  December  2009. 

17.  L.  Ning,  T.T.  Georgiou,  and  A.  Tannenbaum,  “High  resolution  analysis  via  sparsity-inducing 
techniques:  spectral  lines  in  colored  noise,”  Intern.  Symposium  on  Mathematical  Theory  of 
Networks  and  Systems,  Hungary,  July  2010. 

18.  D.  R.udoy  and  T.T.  Georgiou,  Nonstationary  Processes  and  Spectral  Distances,  Intern.  Sym¬ 
posium  on  Mathematical  Theory  of  Networks  and  Systems,  Hungary,  July  2010. 

19.  F.  Carli  and  T.T.  Georgiou  “On  the  Covariance  Completion  Problem  under  a  Circulant  Struc¬ 
ture,”  Intern.  Symposium  on  Mathematical  Theory  of  Networks  and  Systems,  Hungary,  July 
2010. 

20.  P.  Stoica,  L.  Du,  J.  Li,  Tryphon  Georgiou,  “A  New  Method  for  Moving-Average  Parameter 
Estimation,”  44th  Asilomar  Conference  on  Signals,  Systems  and  Computers,  Asilomar  Con¬ 
ference  Grounds,  Pacific  Grove,  California,  November  2010. 

21.  M.  Jovanovic  and  T.  Georgiou,  “Reproducing  second  order  statistics  of  turbulent  flows  using 
linearized  Navier-Stokes  equations  with  forcing,”  Bulletin  of  the  American  Physical  Society, 
63rd  Annual  Meeting  of  the  APS  Division  of  Fluid  Dynamics,  vol.  55(16),  (abstract),  2010. 

22.  D.  Li,  N.  Hovakimyan,  T.T.  Georgiou,  Robustness  of  LI  Adaptive  Controllers  in  the  Gap 
Metric  in  the  Presence  of  Nonzero  Initialization,  IEEE  Conference  on  Decision  and  Control, 
Atlanta,  pages  2723  -  2728,  December  2010. 
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23.  D.  Li,  N.  Hovakimyan,  T.T.  Georgiou,  ’’Robustness  of  LI  adaptive  controllers  in  the  gap 
metric,”  American  Control  Conference  (ACC),  pages  3247  -  3252,  2010. 

24.  L.  Ning,  T.T.  Georgiou  and  A.  Tannenbaum,  Separation  of  system  dynamics  and  line  spectra 
via  sparse  representation,  IEEE  Conference  on  Decision  and  Control,  Atlanta,  pages:  473  - 
478,  December  2010. 

25.  E.  Tannenbaum,  T.T.  Georgiou,  A.  Tannenbaum,  Signals  &  Control  Aspects  of  Optimal  Mass 
Transport  and  the  Boltzmann  Entropy,  IEEE  Conference  on  Decision  and  Control,  Atlanta, 
1885  -  1890,  December  2010. 

26.  L.  Ning  and  T.T.  Georgiou,  Sparse  factor  analysis  via  likelihood  and  ^-regularization,  IEEE 
Conference  on  Decision  and  Control,  Orlando,  Florida,  December  2011. 

27.  Xianhua  Jiang,  Lipeng  Ning  and  Tryphon  T.  Georgiou,  “Geometric  methods  for  structured 
covariance  estimation,”  American  Control  Conf.,  to  appear,  2012. 

2.4  Ph.D.  Thesis 

28.  “Geometric  Methods  for  Spectral  Analysis,”  Xianhua  Jiang,  University  of  Minneosota,  De¬ 
partment  of  Electrical  and  Computer  Engineering,  November  2011. 

2.5  Software 

http:/ /www.ece.umn.edu/'  georgiou/files/software.html 

http:/ /www.ece.umn.edu/”  georgiou/files/HRTSA/download.html 
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3  Accomplished  Research 


The  project  succeeded  at  developing  theory  and  tools  for  high-resolution  analysis,  for  use  in  sensor 
networks,  data  mining,  and  spectral  analysis.  A  key  mathematical  challenge  was  to  introduce  a 
suitable  framework  where  questions  regarding  spectral  uncertainty  and  the  resolution  of  algorithms 
can  be  rigorously  addressed.  To  this  end,  a  geometric  framework  was  developed  based  on  a  family 
of  metrics  which  quantify  distance  between  power  spectral  distributions  in  multivariable  time-series. 
The  geometric  insights  gained  suggested  new  ways  to  aggregate  data  and  refine  various  estimates 
from  disparate  data  sets  (e.g.,  obtained  from  a  variety  of  sensors)  using  geometric  means.  They 
also  suggested  new  techniques  in  filtering,  estimation,  and  modeling.  In  particular,  the  concepts  of 
geodesics  provides  models  for  time- varying  power  spectra  and  the  concept  of  a  geometric  mean,  with 
respect  to  the  various  metrics,  has  been  evaluated  in  the  context  of  estimating  statistical  quantities 
and  model  parameters.  Examples  and  case  studies  of  applying  these  concepts  and  relevant  software 
are  presented  in  the  research  publications.  A  description  of  the  key  concepts  and  of  the  software  for 
high-resolution  analysis  that  has  emerged,  building  on  earlier  efforts  and  along  with  examples,  is  pro¬ 
vided  and  maintained  at  the  Pi’s  website  (http://www.ece. umn.edu/'  georgiou/files/software.html). 


3.1  Geometric  methods  for  spectral  analysis 

Most  promising  concepts  that  have  emerged  from  this  project  include  the  notions  of  geodesics  and 
geometric  means  in  the  space  of  power  spectral  distributions.  Geodesics  are  natural  models  for 
modeling  time- variability  of  power  distributions.  Paths  on  the  space  of  distributions  via  geodesics 
can  be  constructed  in  complete  analogy  with  Gauss’  “least-squares”  well-known  and  ubiquitous 
technique.  The  use  of  geodesics  for  modeling  time-variability  of  power  distribution  of  a  process, 
provides  significantly  higher  resolution.  Geometric  means  on  the  other  hand  are  natural  in  fusing 
together  spectral  information  from  various  sources. 

The  research  has  succeeded  in  producing  a  geometric  framework  for  multivariable  time-series. 
This  relies  on  suitable  notion  of  distance.  There  are  broadly  two  different  rationale  that  lead  to 
consistent,  albeit  different,  geometries  for  power  spectra.  The  first  quantifies  the  ability  to  predict 
a  time  series  using  perturbed  models.  It  leads  to  a  Riemannian  geometry  for  matrix-valued  power 
spectra  (corresponding  to  vector- valued  time-series) .  The  second  is  based  on  the  Monge-Kantorovich 
transportation  theory.  In  either  direction,  the  definition  and  properties  of  the  respective  metrics, 
their  geodesics,  means,  and  relevant  theory  are  being  explained  in  Research  Publications  [1,  6,  7, 
8,  10,  11,  14,  18,  27,  28].  Software  tools  (Matlab  code)  for  and  representative  examples  have  been 
posted  at  the  Pi’s  website: 

http:/ /www.ece.umn.edu/~  georgiou/files/software.html 
http://www.ece. umn.edu/"  georgiou/files/HRTSA/download.html 

The  family  of  metrics  relate  in  unexpected  ways  to  other,  more  traditional,  information-theoretic 
concepts.  More  specifically,  in  [10]  a  relationship  between  the  Kullback-Leibler  divergence  and 
the  Rao  metric,  and  of  the  Bures  metric  from  quantum  mechanics,  the  Hcllinger  distance,  and 
the  Wasserstein  distance  in  optimal  transport  are  linked  together.  In  particular,  the  Wasserstein 
distance  between  Gaussian  distributions  can  be  efficiently  computed  in  many  dimensions  via  solving 
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a  convex  optimization  problem.  The  interplay  between  these  metrics  have  studied  as  a  tool  for 
estimating  statistics  and  parameter,  in  modeling  time-series.  More  specifically,  they  are  shown 
to  have  advantages  in  estimating  structured  covariance  statistics  [10]  -  a  challenging  problem  in 
statistics  (in  shrinkage,  instrumental  variables,  and  error-in- variables  models).  Relevant  software 
and  their  description  along  with  examples  are  also  provided  in  the  aforementioned  websites.  It  is 
also,  a  key  contribution  in  a  recently  completed  Ph.D.  thesis  (by  Ms.  Xianhua  Jiang  in  [28])  and 
presented  in  [10]. 


3.2  Sparse  and  low-rank  models 

Research  publications  [12,  14]  develop  compressive  sensing  techniques  for  system  identification  and 
spectral  analysis.  In  particular,  techniques  from  robust  statistics  (LASSO,  basis  pursuit)  are  suitably 
adapted  to  solve  problems  in  blind  source  deconvolution  and  in  estimation  of  signals-in-noise,  using 
sparse  representation  techniques.  These  can  be  used  as  an  alternative  to  earlier  high-resolution  tools 
(detailed  in  the  Pi’s  website).  This  study  was  done  in  addition  to  the  initially  proposed  plan  for  the 
project,  and  it  was  intended  to  investigate  the  merits  of  this  new  field.  It  was  found  that  they  are 
effective  only  when  the  very  specific  case  where  power  spectra  contain  distinct  lines  (which  are  of 
interest).  Robustness  and  reliability  in  these  are  often  difficult  to  ascertain. 

3.3  Fundamentals  of  feedback  control 

Research  publication  [9]  revisited  and  extended  a  fundamental  result  of  stochastic  optimal  control, 
that  of  the  separation  principle.  The  extension  included  stochastic  processes  which  fall  outside  the 
class  of  the  standard  Gaussian  case  It  was  shown  that  “separation”  of  optimal-filtering  and  optimal- 
control  is  still  valid  when  processes  are  certain  type  of  semi-martingales  with  possible  jumps.  These 
results  provided  new  insights  to  the  control  of  systems  driven  by  discontinuous  processes  modeling 
potential  system-failures. 


4  Personnel 


Tryphon  T.  Georgiou,  Professor 

Ms.  Xianhua  Jiang,  PhD  student  (received  her  PhD  degree  in  2011). 
Mr.  Lipeng  Ning,  PhD  student. 


5  Awards/Recognitions 

The  principle  investigator,  Tryphon  Georgiou,  was  elected  a  Foreign  Member  of  the  Royal  Swedish 
Academy  of  Engineering  Sciences,  (IVA)  in  June  2011. 
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